Cohesion and coherence in the presentation of machine translation products

Different degrees of human intervention can be applied to the preparation of machine translation (MT) products in readying them for their ultimate use. It is suggested here that differences in levels of postediting are associated to some extent with cohesion and coherence. The posteditor is essentially an interpreter of discourse: much of the postediting task involves either employing devices to ensure that the surface pieces of the discourse are connected in meaningful ways or else adjusting the reading of each item against the interpretation of others until the entire underlying text is made to cohere. The presence of cohesion and coherence is examined in three versions of the same machine translation: the raw output, a lightly postedited version, and the final, fully postedited product. 1. Perspective on MT postediting. Machine translation can be delivered the end user as raw output, of course, and it can also be postedited to varying degrees. There is much to be learned from a look at the doctoring that is done when the machine stops working and the human user takes over. Certainly a linguistic study of intervention in the MT product at different levels of refinement can help us to prioritize our strategies. By stratifying the types of corrections that are made, we can begin to orient postediting policy so that today's MT systems are used more effectively, and we can also contribute to the improved performance of the systems of tomorrow. In practice, time and cost constraints often lead to situations in which postediting is curtailed to one degree or another. Depending on the purpose of the translation, nuancing may be traded off for expediency and economy, The most drastic curtailment, of course, is no postediting at all, as is sometimes the policy with translations for information only. Usually, however, even with informative translations there is some type of human intervention. Newman (1988), based on experiments with the SYSTRAN and LOGOS MT systems, has recommended limiting information-only postediting to the replacement of foreign words—words not found in the MT dictionary. Somewhat more intervention is practiced at the U.S. Air Force Foreign Technology Division, where 'partial postediting' addresses seven types of target errors (Bostad 1987). SYSTRAN's Russian-English translations are passed through an automatic postprocessor (EDITSYS) which produces warning flags; whenever any of the seven types of error occurs in the output, the human operator is alerted by a flashing line across the screen. The corrections elicited by EDITSYS will affect, on average, about 20% of the output (Bostad 1987:438). As with most 'information-only' translation, the material handled by the Air Force covers a broad range of subjects and comes 90 / Georgetown University Round Table on Languages and Linguistics 1989 from a wide range of sources. This is the opposite of constrained input in a highly limited domain, where MT systems may be able to handle most problems at the level of the algorithm and generate a usable translation that requires very little correction. This latter situation is exemplified by METEO 2, the system that translates Canadian weather forecasts; as of mid-1988, interventions were down to the point that only 3.4% of the text was being affected (Chandioux p.c.). None of these applications, however, is what you might call 'mainstream' translation. In the everyday world, by far the greatest demand is for translations of general and technical material that leave no doubt as to the meanings intended by the original author. To produce such translations, given the current state of MT art, may require a somewhat more intensive human review than what has just been described. Still, it would be useful to distinguish levels of 'light' and 'full' postediting. At the European Commission in Luxembourg, for example, 'rapid' postediting of SYSTRAN machine translations was sanctioned under a project launched in May 1982 (Wagner 1983). Differences between rapid and conventional postediting were later examined by Löffler-Laurian (1986). For our work at the Pan American Health Organization (PAHO), where we have been postediting machine translation for nearly ten years, it would be of great practical value to identify the differences between two such levels and to systematize their implementation. Moreover, to the extent that we are able to relate the distinctions to broad linguistic principles, our findings may be of more general interest for MT development and for translation theory as well. 2 Approach to a definition of levels. My hypothesis is that the differences in translation 'quality at the levels of rapid and full postediting can be correlated, at least partially, with syntactic corrections and degrees of COHESION, on the one hand, and COHERENCE on the other. Of course, raw MT output, as well as that which is checked only minimally for predictable trouble spots, is bound to have some syntactic problems that need to be corrected. Once these have been dealt with, it is likely that the product may still be further improved: cohesive devices can be introduced that will establish clearer connections between the pieces, and more refined interpretations can be made of the nuances that help to convey the author's intentions to the target audience. For the rapid postedit, Löffler-Laurian (1983) proposes that revision should concentrate on 'vocabulary changes', especially in domains for which the dictionary has not been highly developed: translations should be supplied for not-found words, and erroneous glosses should be corrected. Also, passages that are incomprehensible should be repaired. These are useful criteria. In addition, on the basis of our experience at PAHO, I would say that at this level many devices can be invoked which will enhance the cohesiveness of the text. The full postedit, in turn, involves modifications that will bring out nuances and enable the reader to grasp the complete significance of the text. It makes the difference between a translation that is merely passable and one that is appropriate for the most demanding of circumstances. Löffler-Laurian (1983) has offered a set of four guidelines and twelve specific rules for the Muriel Vasconcellos / 91 posteditor working at this level. Here the PAHO experience points, further, to interpretations leading to improved coherence. In testing the hypothesis about the respective roles of cohesion and coherence, it is important that we work from a clear definition of each of these terms. 3 Concepts. Widdowson (1979:87) defines cohesion as 'the overt structural link between sentences as formal items', and coherence as 'the link between the communicative acts that sentences are used to perform'. He goes on to suggest that cohesion is the propositional relation between the parts of a discourse, whereas coherence is the illocutionary relation. For present purposes, the definitions of both concepts have been broadened to apply to relations within sentences or communicative acts as well as between them. Thus, cohesion is taken to refer to ties between elements manifest in the surface structure of the discourse, while coherence has to do with the interpretation of connectedness in the underlying text. 3.1 Cohesion. Cohesion is easier to describe than coherence, and easier to recognize. The devices can be specified, and when applied to translation they can yield considerable payoff in terms of understandability. According to Halliday and Hasan (1976:4), cohesion occurs when an element in discourse cannot be effectively decoded without invoking another element in the text or the discourse situation. 'When this happens, a relation of cohesion is set up, and the two elements, the presupposing and the presupposed, are thereby at least potentially integrated into a text'. They identify the following types: reference, substitution/ellipsis, lexical cohesion, and conjunction. Since these headings will be applied below to some of the corrections that are made in MT postediting, they are elaborated here in some detail. In the case of reference, an element in discourse relies on some other element for its interpretation: information must be recovered about it--either a referential meaning or the identity of a particular thing--in order for it to be decoded. Personal and possessive pronouns, for example, set up pronominal reference. Demonstrative reference is established by demonstrative pronouns and also by the definite article the. Comparative reference involves identity, similarity, difference, or quantitative or qualitative relations between discourse entries. The referent may be present in the discourse situation rather than the text, in which case the reference is exophoric. Cohesion is created by the fact that the same concept enters the discourse a second time, and the cohesive tie is the connection between the two occurrences (31). Substitution is 'the replacement of one item by another' (88). The second item, or substitute, establishes a cohesive link with the first. Whereas reference is a relationship between meanings, substitution is a relationship between linguistic items. The substitute is used to avoid repetition. In English, NPs can be replaced by one(s) or same; verbs by do (+so/it/that/the same/likewise), be (+so/it/that), have (+to); and clauses by so or not. Unlike reference, substitution cannot be exophoric; it can only involve the elements expressed in the discourse proper. 92 / Georgetown University Round Table on Languages and Linguistics 1989 Ellipsis (142) may be seen as a special subtype of substitution in which a linguistic item is replaced by nothing. There is cohesion with the zero element in the same way as there is with the substitute. Lexical cohesion (274) connects discourse entries not through grammatical resources, as above, but rather through lexical choice. A synonym, a broader or narrower term, or a related term revives a concept in the discourse. There is also cohesion between any pair of lexical items that belong to the same ordered set or paradigm (Tuesday...Thursday, north...south) and between items that often cooccur--for example, blade...sharp, garden...dig, try...succeed, king...crown, boat...row (285). Conjunction is different from the other four types of cohesion because attention is focused on the meaning of the cohesive relation itself rather than on the elements that are tied together (226-227). There is a large inventory of cohesive relations under the broad heading of conjunction, and the authors have classified them according to their function: additive ('and', 'or else', 'furthermore', 'for instance', 'similarly', 'on the other hand'), adversative ('but', 'nevertheless', 'in fact', 'on the other hand', 'instead', 'rather', 'in any case'), causal ('therefore', 'with this in mind', 'it follows', 'in that case', 'otherwise'), and temporal ('next', 'at once', 'meanwhile', 'finally', 'up to now', 'in short'). Presumably the authors' lists could be expanded to include such discourse markers as 'oh', 'well', 'y'know', 'I mean' (Schiffrin 1987). In addition to using the foregoing devices, which can be formulated quite explicitly on the basis of grammatical and lexical properties, cohesion involves developing the overall fabric of the text through the distribution of new and old information and through the staging effect created by message themes (Halliday and Hasan 325, Halliday 1967-68, Vasconcellos 1985,1986a, 1986b). 3.2 Coherence. Unlike cohesion, coherence underlies the discourse and has no predictable reflex in surface structure. Whereas cohesion has to do with relations between surface linguistic forms and between propositions, coherence involves connectedness within the communication act itself. The speaker/writer is now seen as communicator, and the listener/reader as interpreter. The progress of a discourse is determined by the communicator's choices of meanings to be focused on. In turn, the interpreter of a discourse (in our case the posteditor) must be able to decide for each entry in the discourse which meaning type, and within it which specific meaning among possible alternatives, is intended. If the posteditor's interpretation matches the author's intention, the translation is fully successful—although in reality this success is apt to be achieved only to an approximate degree. Communicators and interpreters assume that a text is coherent. Coherence is observed, and therefore defined, more through its absence than its presence. Lack of coherence may be illustrated by the following example (van Dijk 1972): (1) We will have guests for lunch. Calderón was a great Spanish writer. Muriel Vasconcellos / 93 Despite the strong tendency to assume coherence, it is difficult for an interpreter to see any connectedness between the two entries. Coherence is present, on the other hand, in a similar sequence: (2) You ought to read Wombats Galore. Bruce McQuarrie is a great author. Even though in fact it is nonsense contrived precisely to make this point (Stubbs 1983:124). Some authors would assign part of semantic connectedness to cohesion, but the position taken here is that coherence, rather than cohesion, underlies the interpretations of textual meaning. For Sanders (1987), it is coherence which provides the communicator with the cognitive basis for formulating discourse entries so that control is exercised over the way he or she is understood (7). From the perspective of the interpreter, who in the case of written text is distanced from the author at least in time if not in space, readings have to be adjusted back and forth as the discourse unfolds until each discourse entry has a specific interpretation that fits with what went before (84). A.single word may constitute a discourse entry, and its reading has to be adjusted against others in the context until they are made to cohere as much as possible. Sanders illustrates this process with a well-known sentence: (3) Colorless green ideas sleep furiously. At first we are struck by blatant incoherence. This is because we give an unmarked interpretation to each of the terms. However, if we force ourselves to assume that the message is coherent, we can try to read different meanings into the components until a coherent interpretation of the whole is arrived at. Each term is examined for its range of possible meanings, and the various options are tested against the surrounding context. To start with, by looking ahead we know that colorless does not readily apply to the upcoming concepts of green and ideas, so we backtrack and interpret it as 'lackluster'. We then rethink the meaning of green and reject the more usual one of 'a color' in favor of 'unripened'. And so on. Sanders' result is: (3') Lackluster unripened ideas lying dormant are volatile. The process that Sanders describes is constantly exercised in translation, especially in the postediting of machine output. Postediting is an ongoing process of interpretation, since the pieces of the target language are already given. The job of the posteditor is to examine these pieces, make a 'specific interpretation' of the meaning intended by the author, and adjust the wording so that the text becomes more coherent. The computer can and often does generate a set of pieces which an interpreter can appreciate as a fully understandable translation which is both grammatically and discoursally well-formed. But this judgment has to be made by the posteditor, and such sentences may be intermixed with others that are less felicitous. 94 / Georgetown University Round Table on Languages and Linguistics 1989 4 Cohesion and coherence in postediting. In our work at the Pan American Health Organization with Spanish-English and English-Spanish MT (SPANAM and ENGSPAN, respectively), we have found that a conventional postedit is usually accomplished in two passes: a first 'rough draft' and then a final polishing. This has been our typical experience over the last ten years. We would like to know now whether these two passes could be made to correspond to linguistically describable levels. What is really done in each of the two passes? Are there linguistic criteria that distinguish one level from the other? And finally, are there purposes for which raw MT and first-pass MT are adequate? Up to now the answers to these questions have eluded us. No one has been able to tell posteditors exactly what to look for. Of course, with many of the changes that are introduced there is general agreement on the need for something to be done—if not on the solution. But with other changes there is debate about whether they are essential or even worthwhile at all. So far, it has been difficult to clarify how they contribute to making the translation more explicit--and therefore more useful. To address these questions, I singled out one of the jobs in our regular Spanish-English production stream for which it was possible to reconstruct three different versions: the raw output (presented in side-by-side form at Appendix A and in target-only form as Appendix B), a first-pass postedit (Appendix C), and the final translation (Appendix D). The complete text was a 7,000-word report on the status of nutrition in Latin America, a subject on which SPANAM has often been exercised in the past. The first 312 words, which are fairly typical of the rest of the document, were examined in depth and are discussed at each of the two levels in the following sections. 4.1 The first pass. For both the first pass and the final translation, the changes that had been made were grouped under three broad headings: syntactic corrections, cohesive devices, and interpretations for coherence. An effort was made to assign all the changes, including lexical choices, to one of another of these categories. At the level of the first pass, the posteditor made a total of 33 changes (shown in Appendix B)., The distribution was as follows: Syntactic corrections 9 27% Cohesive devices 21 64% Interpretations for coherence 3 9% 100% Syntactic corrections. Three of the syntactic changes were merely punctuation: one a comma to mark a nonrestrictive relative clause [line 13], another a comma to match an existing comma for a parenthetical phrase [line 17], and the last a hyphen [line 22]. Two others also involved the further marking of a nonrestrictive relative clause: omission of and as a translation of the Spanish clause-marker y and substitution of which for that [both on line 13]. Two were corrections in prepositional government (place demands on [line 13] and suffer from [line 21]). An adjustment was made to accommodate the fact that contribute in English cannot be followed by an infinitive [line 16], Muriel Vasconcellos / 95 The last correction was a VSO construction that could not easily be 'quick-fixed' (see Vasconcellos 1986a) and required the movement of four words to the end of the sentence. Of the nine syntactic corrections, two (22%) were made using macros, indicating that these were operations commonly performed by posteditors. (Use of a macro, of course, speeds up the process.) Cohesive devices. Of the 21 cohesive devices, 12 (57%) had to do with definiteness, a subcategory of referential cohesion: two called for insertion of the definite article and nine for its deletion (the changes on line 2 were counted twice, once as deletion of the article and once as conjunction). Seven of the other nine devices could be accounted for in terms of conjunction. In five instances, conjoining of the terms in an enumeration was highlighted cohesively by repetition of the preposition [lines 2 (twice), 5,6] or downgraded by the deletion thereof [line 20]. In another case [line 9], a relative clause marker was changed from a comma to a dash, giving more independence to the conjoining relation. Also under conjunction, the head noun capacity was redundant in the premodifying enumeration of the NP whose head was performance. The other two changes had to do with discourse texture. In one, the information structure was preserved by postposing the concept disadvantaged [line 7] after populations. In the other, movement of the word usually (from the Spanish generalmente) to the front of the clause gave it thematic status [line 10]. Of the 21 cohesive devices, 12 (57%) were introduced using macros. There were also interpretations for coherence at this level. The word exist [line 7] emphasizes the notion of 'existence' in a context where it does not apply. Changing the translation usually to in general [line 10, counted previously as a move for purposes of thematization] brackets the clause that follows and appears to approximate more closely the meaning originally intended. Finally, deletion of aspects of seems to tighten the coherence in English. 4.2 Final translation. At this maximum level of refinement 14 additional changes were made, which showed the following breakdown: Syntactic corrections 0 0% Cohesive devices (?) 3 21% Interpretations for coherence 11 79% 100% As it can be seen, there was a clear preponderance of interpretations with a view to improving coherence. Only three of the changes could be regarded as cohesive devices, and in each case an underlying motivation of coherence could be argued. One of the changes that was classified as cohesive was the replacement of however by nevertheless [line 3]. In surface structure, this is a cohesive relation expressed through a conjunction. On the other hand, the interpretation that led to the change might well be considered to involve coherence. This was also true of the changes in the conjunctions from in 96 / Georgetown University Round Table on Languages and Linguistics 1989 general to in sum [line 10]. The other candidate cohesive device was the use of commas to bracket the phrase on lines 24-25. Again, although the device is a surface-structure mechanism, one could argue that it was necessitated by the expansion which had been added between the commas. Eight (73%) of the remaining 11 changes were clear-cut expansions beyond the propositional content given by the machine translation [two insertions on line 7 plus those on lines 16, 17, 18, 24, 26, and 27]. Because the material had not been present in the discourse, these changes could only be classed as interpretations for coherence. Two others [lines 10 and 40], although they did not expand the number of words in the text, added further semantic specificity which had not been there before, and in this sense they were also expansions. Finally, the use of to [line 28] adds force to the claim being made. 5 Discussion. The foregoing analysis bears out the difficulty of separating cohesion and coherence. In several of the examples it seemed that even though cohesive devices had been used, because of the circumstances of postediting there was also a strong component of interpretation for coherence. For instance, the changing of however to nevertheless [line 3] and in general to in sum [line 10] were both further refinements of cohesive relations that were already present in the discourse--and in fact had been introduced during the first pass. It looks as if two different types of motivation were at work, In the first pass, the need for a cohesive tie was detected, and the material introduced was a close approximation of the original Spanish. In the final polishing, however, the posteditor became interpreter and proceeded to introduce semantic components which represented a slight departure from the unmarked meaning of these conjunctions, doing so in the interest of coherence. What have we learned from this exercise? In the sample studied it was clear that syntactic corrections and cohesive devices predominated in the first pass and that interpretations for coherence accounted for the changes in the final translation. In the haste of work, the distinction between these two levels tends to blur: during the first pass it may happen that interpretations are introduced, while in the final review action may be taken on opportunities that were missed the first time around. It is not reasonable to expect that posteditors will follow a rigorous separation between the two. Still, time can be saved for some applications if an effort is made to limit changes to syntactic corrections and cohesive devices. As far as the contribution to MT development is concerned, it is reasonable to hope that many cohesive devices can eventually be written into basic algorithms or interor postprocessors. On the other hand, it is also important to recognize the posteditor's role as interpreter of coherence, and to understand that this aspect of human performance is beyond formalization, Muriel Vasconcellos / 97 98 / Georgetown University Round Table on Languages and Linguistics 1989] Muriel Vasconcellos / 99 100 / Georgetown University Round Table on Languages and Linguistics 1989 Muriel Vasconcellos / 101